feat(frontend): Support for UDF Ui Parameter by carloea2 · Pull Request #4268 · apache/texera

carloea2 · 2026-03-07T09:12:40Z

What changes were proposed in this PR?

Add Python UDF Parameter support:

What the user writes (Python)

Users declare UI parameters once in open(), and then use the typed value directly:

from pytexera import *
from typing import Iterator, Optional

class ProcessTupleOperator(UDFOperatorV2):

    @overrides
    def open(self):
        # declare UiParameters once, store the parsed runtime values
        self.value1 = self.UiParameter(name="param1", type=AttributeType.DOUBLE).value
        self.value2 = self.UiParameter(name="param2", type=AttributeType.INT).value
        self.value3 = self.UiParameter(name="param3", type=AttributeType.STRING).value
        self.value4 = self.UiParameter(name="param4", type=AttributeType.TIMESTAMP).value

    @overrides
    def process_tuple(self, tuple_: Tuple, port: int) -> Iterator[Optional[TupleLike]]:
        print(self.value1)
        print(self.value2)
        print(self.value3)
        print(self.value4)
        yield tuple_

What shows up in the UI

From those self.UiParameter(...) lines, the property panel automatically generates a Parameters section with one row per parameter:

Name + Type are read-only (so they stay consistent with the code)
Value is editable (so users can change runtime values without touching the script)

How the values get into Python

When the workflow runs, we inject the UI values into the UDF and the base class applies them right before open() executes. That way, when the user calls UiParameter(...).value, they get the current value from the UI.

Overall:

sequenceDiagram
    autonumber

    participant FE as Frontend UI
    participant Parser as Parser Service
    participant Store as Operator Properties
    participant BE as Scala Backend
    participant Inj as Python Code Injector
    participant Py as Python Runtime
    participant UDF as User UDF

    FE->>Parser: Send Python code
    Parser->>Parser: Extract self.UiParameter declarations
    Parser-->>FE: Return parameter schema

    FE->>Store: Save uiParameters<br/>name, type, value
    Store->>BE: Submit operator descriptor

    BE->>Inj: Pass original code + uiParameters
    Inj->>Inj: Validate names and types
    Inj->>Inj: Generate injected hook
    Inj-->>BE: Return modified Python code

    BE->>Py: Execute injected code
    Py->>UDF: Load UDF class
    UDF->>Py: open() is wrapped

    Py->>UDF: Call injected hook
    UDF-->>Py: Return parameter values

    Py->>UDF: Provide typed UiParameter values
    UDF->>Py: Execute workflow logic

Detailed:

sequenceDiagram
    autonumber

    actor User

    participant CodeEditor as CodeEditorComponent<br/>Monaco editor
    participant YCode as Yjs shared code
    participant Sync as UiUdfParametersSyncService
    participant Parser as UiUdfParametersParserService
    participant ParamUI as UiUdfParametersComponent
    participant Formly as Formly / Operator Properties
    participant OpDesc as Python UDF Operator Descriptor<br/>Scala backend
    participant Injector as PythonUdfUiParameterInjector
    participant Runtime as Python Runtime
    participant Support as _UiParameterSupport
    participant UDF as User UDF Class

    User->>CodeEditor: Writes Python UDF code
    CodeEditor->>YCode: Updates shared code text
    YCode->>Sync: Emits code-change event

    Sync->>Parser: Parse latest Python code
    Parser->>Parser: Find supported UDF class
    Parser->>Parser: Scan self.UiParameter(...)
    Parser-->>Sync: Return inferred parameters<br/>name + AttributeType

    Sync->>Formly: Update operator uiParameters structure
    Formly->>ParamUI: Render parameter rows
    ParamUI-->>User: Shows inferred params<br/>name/type locked, value editable

    User->>ParamUI: Enters parameter values
    ParamUI->>Formly: Store edited values
    Formly->>OpDesc: Persist uiParameters in operator properties

    User->>OpDesc: Runs workflow

    OpDesc->>Injector: inject(code, uiParameters)
    Injector->>Injector: Validate uiParameters
    Injector->>Injector: Build parameter map
    Injector->>Injector: Generate reserved hook method
    Injector->>Injector: Insert hook into user UDF class
    Injector-->>OpDesc: Return injected Python code

    OpDesc->>Runtime: Execute OpExecWithCode(injectedCode, "python")
    Runtime->>UDF: Load user UDF class
    UDF->>Support: __init_subclass__ wraps open()

    Runtime->>UDF: Start operator execution
    UDF->>Support: wrapped open() runs first
    Support->>UDF: Call _texera_injected_ui_parameters()
    UDF-->>Support: Return UI parameter values
    Support->>Support: Store injected values

    Support->>UDF: Continue original user open()
    UDF->>Support: Create/read self.UiParameter(...)
    Support->>Support: Look up injected value by name
    Support->>Support: Convert value using AttributeType
    Support-->>UDF: Return typed Python value

    UDF->>Runtime: Process tuples / batches / source output

Any related issues, documentation, discussions?

Closes #4154

How was this PR tested?

Testing added to backend and frontend

Was this PR authored or co-authored using generative AI tooling?

Co-generated with GPT

…into feat/ui-parameter

chenlica · 2026-03-07T16:06:20Z

@aglinxinyuan Please review it.

Copilot

Pull request overview

This PR adds UDF UI Parameter support, allowing users to declare typed parameters in their Python UDF open() method via self.UiParameter(name=..., type=AttributeType....). The frontend parses these from the code, displays them as an editable properties panel, and the backend injects the UI-supplied values into the Python code before execution.

Changes:

Frontend: New UiUdfParametersParserService and UiUdfParametersSyncService that parse self.UiParameter(...) calls from Python code and sync parsed parameters to the operator property store; new UiUdfParametersComponent renders the parameters table in the property panel.
Backend (Scala): New UiUDFParameter model and PythonUdfUiParameterInjector that injects a _texera_injected_ui_parameters hook method into the UDF class with the UI-supplied values encoded via the pybuilder mechanism; all three Python UDF operator descriptors updated to run the injector.
Backend (Python): New _UiParameterSupport mixin class added to all UDF base classes; it wraps open() via __init_subclass__ to apply injected values before user code runs.

Reviewed changes

Copilot reviewed 24 out of 25 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
`workflow-compiling.interface.ts`	Adds `large_binary` to `AttributeType`, JAVA/Python attribute type name constants, and derived union types for cross-language type token handling
`ui-udf-parameters-parser.service.ts`	New service that parses `self.UiParameter(...)` calls from Python code and normalizes type tokens to canonical schema names
`ui-udf-parameters-parser.service.spec.ts`	Tests for the parser service; contains incorrect expected type values
`ui-udf-parameters-sync.service.ts`	New service that attaches to YText changes and syncs parsed parameter structure to operator properties
`ui-udf-parameters.component.*`	New Angular Formly custom type component rendering the parameters table with read-only name/type and editable value columns
`operator-property-edit-frame.component.ts/.scss`	Subscribes to param changes and maps the `uiParameters` field key to the custom Formly type; adds styling
`code-editor.component.ts`	Attaches/detaches the YText listener when the Monaco editor is initialized/destroyed
`formly-config.ts` / `app.module.ts`	Registers the new `ui-udf-parameters` Formly type and declares the component
`PythonUdfUiParameterInjector.scala`	New Scala object that injects the `_texera_injected_ui_parameters` hook method into UDF classes
`PythonUdfUiParameterInjectorSpec.scala`	Tests for the Scala injector; contains failing test assertions
`UiUDFParameter.scala`	New Scala model class for a UI parameter (attribute + string value)
`PythonUDFOpDescV2.scala` / `DualInputPortsPythonUDFOpDescV2.scala` / `PythonUDFSourceOpDescV2.scala`	Adds `uiParameters` field and wires the injector call
`Attribute.java`	Adds `@EncodableStringAnnotation` to `getName()` for safe encoding in pybuilder templates; introduces unused imports
`udf_operator.py`	Adds `_UiParameterSupport` mixin with `UiParameter` inner class and wrapping mechanism
`attribute_type.py`	Adds `FROM_STRING_PARSER_MAPPING` for string-to-type conversion
`pytexera/__init__.py` / `pyamber/__init__.py`	Exports `AttributeType`
`collab-wrapper.component.css`	Minor whitespace change to an already-invalid CSS property

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

aglinxinyuan

Please fix the test cases, as the frontend currently fails to compile. Also address the comments left by Copilot. The PR still feels quite draft-like, so please take some time to polish it before requesting for review.

carloea2 · 2026-04-12T07:23:22Z

@aglinxinyuan please continue with the review

aglinxinyuan

LGTM in general. Since it's a huge PR, let's have a secondary reviewer on this. @kunwp1, please review it.

Since it's a huge PR, let's have a secondary reviewer on this.

kunwp1

Thanks for the PR! I haven't completed the full code review yet, but I have a few observations regarding the user experience and some edge-case behavior:

Documentation for UI Parameters: Since the new UI parameter introduces a bit of complexity on editing the UDF code, it might be difficult for users to add a parameter without guidance. Could you add some docstrings or comments to the Python UDF operators explaining how to use this parameter?
Data Type Restrictions (BINARY / LARGE_BINARY): I am concerned about the usefulness of supporting BINARY and LARGE_BINARY types here. Since the property panel relies on text input rather than file uploads, providing a raw byte array manually is pretty bad experience for a user. Should we disable these types for now? If you have a specific use case in mind where manual hex/binary input is necessary, let's discuss!
UI Sync Bug: I noticed that if a parameter is commented out in the code, it still persists in the property panel. We should ensure the UI stays in sync with the active code state. Could you take a look at the parsing logic for that?

carloea2 · 2026-04-15T02:50:57Z

@kunwp1

Thanks.

Ok.
I will remove them.
Got it.

chenlica · 2026-04-15T06:29:32Z

@carloea2 Please resolve conflicts.

carloea2 · 2026-04-15T06:36:41Z

@chenlica I will fix it after the current code pass the review, that is more efficient I think.

Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>

aglinxinyuan · 2026-04-15T06:45:57Z

@chenlica I will fix it after the current code pass the review, that is more efficient I think.

I just did the fix since it's caused by my changes. It's usually easier to resolve conflicts earlier than later.

carloea2 · 2026-04-15T06:53:36Z

@chenlica I will fix it after the current code pass the review, that is more efficient I think.

I just did the fix since it's caused by my changes. It's usually easier to resolve conflicts earlier than later.

Ok, thanks.

carloea2 · 2026-04-15T22:27:46Z

@kunwp1

I removed support for Binary/Large Binary
I took care of the parsing issue
I added doc blocks to the UDF, let me know if you want a change.

Thanks.

kunwp1

Left some comments. I tested the PR and the features look good. We need a architecture diagram do the review because the change is very big and I don't follow the details of the implementation. Can you add a diagram in the PR?

kunwp1 · 2026-04-17T20:29:15Z

    "Iterator",
    "Optional",
    "Union",
+    "Dict",


Remove Dict and Any

No, this change is needed for the feature to work. The reason is the definitions of UiParameter use those as the return value types, so since the code comes from the User, I make them available from the import.

@kunwp1 NameError: name 'Dict' is not defined
2026-04-18 12:17:21.586 | ERROR | core.architecture.rpc.async_rpc_server:receive:102 - name 'Dict' is not defined
File "C:\Users\carlo\AppData\Local\Temp\tmpkqz8h60kfsTempFS\udf-v1.py", line 12, in
class ProcessTupleOperator(UDFOperatorV2):
-> <class 'pytexera.udf.udf_operator.UDFOperatorV2'>

File "C:\Users\carlo\AppData\Local\Temp\tmpkqz8h60kfsTempFS\udf-v1.py", line 30, in ProcessTupleOperator
def _texera_injected_ui_parameters(self) -> Dict[str, Any]:

NameError: name 'Dict' is not defined

carloea2 · 2026-04-18T05:18:48Z

@kunwp1 thanks for the review, I will let you know when the diagram is ready, though I think it may not reflect the changes, since the architecture is the same, just changed the way the UDFs contents are generated.

carloea2 · 2026-04-18T05:34:28Z

@kunwp1 are these diagrams ok?:

Overall:

sequenceDiagram
    autonumber

    participant FE as Frontend UI
    participant Parser as Parser Service
    participant Store as Operator Properties
    participant BE as Scala Backend
    participant Inj as Python Code Injector
    participant Py as Python Runtime
    participant UDF as User UDF

    FE->>Parser: Send Python code
    Parser->>Parser: Extract self.UiParameter declarations
    Parser-->>FE: Return parameter schema

    FE->>Store: Save uiParameters<br/>name, type, value
    Store->>BE: Submit operator descriptor

    BE->>Inj: Pass original code + uiParameters
    Inj->>Inj: Validate names and types
    Inj->>Inj: Generate injected hook
    Inj-->>BE: Return modified Python code

    BE->>Py: Execute injected code
    Py->>UDF: Load UDF class
    UDF->>Py: open() is wrapped

    Py->>UDF: Call injected hook
    UDF-->>Py: Return parameter values

    Py->>UDF: Provide typed UiParameter values
    UDF->>Py: Execute workflow logic

Detailed:

sequenceDiagram
    autonumber

    actor User

    participant CodeEditor as CodeEditorComponent<br/>Monaco editor
    participant YCode as Yjs shared code
    participant Sync as UiUdfParametersSyncService
    participant Parser as UiUdfParametersParserService
    participant ParamUI as UiUdfParametersComponent
    participant Formly as Formly / Operator Properties
    participant OpDesc as Python UDF Operator Descriptor<br/>Scala backend
    participant Injector as PythonUdfUiParameterInjector
    participant Runtime as Python Runtime
    participant Support as _UiParameterSupport
    participant UDF as User UDF Class

    User->>CodeEditor: Writes Python UDF code
    CodeEditor->>YCode: Updates shared code text
    YCode->>Sync: Emits code-change event

    Sync->>Parser: Parse latest Python code
    Parser->>Parser: Find supported UDF class
    Parser->>Parser: Scan self.UiParameter(...)
    Parser-->>Sync: Return inferred parameters<br/>name + AttributeType

    Sync->>Formly: Update operator uiParameters structure
    Formly->>ParamUI: Render parameter rows
    ParamUI-->>User: Shows inferred params<br/>name/type locked, value editable

    User->>ParamUI: Enters parameter values
    ParamUI->>Formly: Store edited values
    Formly->>OpDesc: Persist uiParameters in operator properties

    User->>OpDesc: Runs workflow

    OpDesc->>Injector: inject(code, uiParameters)
    Injector->>Injector: Validate uiParameters
    Injector->>Injector: Build parameter map
    Injector->>Injector: Generate reserved hook method
    Injector->>Injector: Insert hook into user UDF class
    Injector-->>OpDesc: Return injected Python code

    OpDesc->>Runtime: Execute OpExecWithCode(injectedCode, "python")
    Runtime->>UDF: Load user UDF class
    UDF->>Support: __init_subclass__ wraps open()

    Runtime->>UDF: Start operator execution
    UDF->>Support: wrapped open() runs first
    Support->>UDF: Call _texera_injected_ui_parameters()
    UDF-->>Support: Return UI parameter values
    Support->>Support: Store injected values

    Support->>UDF: Continue original user open()
    UDF->>Support: Create/read self.UiParameter(...)
    Support->>Support: Look up injected value by name
    Support->>Support: Convert value using AttributeType
    Support-->>UDF: Return typed Python value

    UDF->>Runtime: Process tuples / batches / source output

kunwp1 · 2026-04-18T17:16:32Z

I haven't read the details of the diagram yet but will do it later after all the comments are addressed. Please move the diagrams to the PR description.

carloea2 · 2026-04-18T19:59:38Z

@kunwp1
Thanks. I have resolved your comments. The diagram is also now in the PR description.

carloea2 added 13 commits February 26, 2026 13:46

v1

f206864

Merge branch 'apache:main' into feat/ui-parameter

299dc0f

v2

5b29ad8

Merge branch 'feat/ui-parameter' of https://github.com/carloea2/texera …

6258a99

…into feat/ui-parameter

v2

3550ccd

py2udf

7a210a7

v2.1

4a56861

Merge branch 'feat/ui-parameter' of https://github.com/carloea2/texera …

16f89ac

…into feat/ui-parameter

vnt

5aa3f4e

v2

1d82c9a

v2.1

b36628f

Update ui-udf-parameters.component.scss

dd46609

v2.1

2c8167b

github-actions bot added feature engine python frontend Changes related to the frontend GUI common labels Mar 7, 2026

carloea2 added 3 commits March 7, 2026 03:16

v3

06d7f5a

v3

ac2b265

Merge branch 'main' into feat/ui-parameter

c8239c2

carloea2 changed the title ~~Feat/UI parameter~~ feat(frontend): Support for UDF Ui Parameter Mar 7, 2026

chenlica requested a review from aglinxinyuan March 7, 2026 16:06

Merge branch 'main' into feat/ui-parameter

edd3869

aglinxinyuan requested a review from Copilot March 9, 2026 19:15

Copilot started reviewing on behalf of aglinxinyuan March 9, 2026 19:15 View session

Copilot AI reviewed Mar 9, 2026

View reviewed changes

aglinxinyuan requested changes Mar 9, 2026

View reviewed changes

Merge branch 'main' into feat/ui-parameter

0c9f0cd

aglinxinyuan requested a review from kunwp1 April 13, 2026 00:03

aglinxinyuan previously approved these changes Apr 13, 2026

View reviewed changes

aglinxinyuan self-requested a review April 13, 2026 00:05

kunwp1 reviewed Apr 15, 2026

View reviewed changes

aglinxinyuan added 3 commits April 14, 2026 23:42

Merge branch 'main' into feat/ui-parameter

9d506ad

Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>

Remove FileSelectionComponent from app.module.ts

9f2bdbd

Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>

Update app.module.ts

672ec4b

Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>

aglinxinyuan and others added 2 commits April 14, 2026 23:58

fix fmt

f9eaf4f

v9

4d0107c

chenlica and others added 3 commits April 15, 2026 20:11

Merge branch 'main' into feat/ui-parameter

670ec8c

Merge branch 'main' into feat/ui-parameter

086d781

Merge branch 'main' into feat/ui-parameter

4b2c25d

kunwp1 requested changes Apr 17, 2026

View reviewed changes

Merge branch 'main' into feat/ui-parameter

50e8aa2

carloea2 mentioned this pull request Apr 18, 2026

feat(frontend): Ui Parameter - Boolean Support Dropdown. #4412

Open

v13

606d241

github-actions bot added the dependencies Pull requests that update a dependency file label Apr 18, 2026

Merge branch 'main' into feat/ui-parameter

496de38

Conversation

carloea2 commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this PR?

What the user writes (Python)

What shows up in the UI

How the values get into Python

Any related issues, documentation, discussions?

How was this PR tested?

Was this PR authored or co-authored using generative AI tooling?

Uh oh!

chenlica commented Mar 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aglinxinyuan left a comment

Choose a reason for hiding this comment

Uh oh!

carloea2 commented Apr 12, 2026

Uh oh!

aglinxinyuan left a comment

Choose a reason for hiding this comment

Uh oh!

kunwp1 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

carloea2 commented Apr 15, 2026

Uh oh!

chenlica commented Apr 15, 2026

Uh oh!

carloea2 commented Apr 15, 2026

Uh oh!

aglinxinyuan commented Apr 15, 2026

Uh oh!

carloea2 commented Apr 15, 2026

Uh oh!

carloea2 commented Apr 15, 2026

Uh oh!

kunwp1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kunwp1 Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

carloea2 Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

kunwp1 Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

carloea2 Apr 18, 2026

Choose a reason for hiding this comment

Uh oh!

carloea2 Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

carloea2 commented Apr 18, 2026

Uh oh!

carloea2 commented Mar 7, 2026 •

edited

Loading

kunwp1 left a comment •

edited

Loading

carloea2 Apr 18, 2026 •

edited

Loading